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Introduction: A 2009 National Academies of Sciences report on child mental health prevention 
and treatment concluded that screening for mental health risk is an essential component of service 
delivery. To date, however, there are few practical assessments available or practices in place 
that measure individual child risk, or risk aggregated at the school or community level. This study 
examined the utility of a 30-item paper and pencil student self-report screener of behavioral and 
emotional risk (BER) for surveying community risk among 7 schools. 

Methods: In 2010, 2,222 students in 3 middle and 4 high schools in a medium-sized school district 
in Georgia were administered the Behavioral and Emotional Screening System Self-Report Child/ 
Adolescent form (BESS Student). The BESS is designed to measure 4 sub-syndromal BER factors 
for developing mental health disorders: inattention/hyperactivity, internalizing, school problems, 
and personal adjustment. Analysis of Variance and Chi Square analyses were used to assess the 
association between adolescent self-reported BER as an indicator of school BER, grade level, child 
ethnic identification and gender, socioeconomic status, and special education placement status. 

Results: BESS scores differentiated well between schools for overall BER and special education 
status, as well as between grade levels, ethnicity, and gender groups. One high school, known by 
the school administration to have numerous incidents of student behavior problems, had the most 
deviant 4 BER domain scores of all 7 schools. Girls rated themselves as having a higher prevalence 
of BER (14%) than boys (12%); middle school students reported fewer difficulties than high school 
students. 

Conclusion: Middle and high school students were capable of identifying significant differences in 
their own BER across schools, suggesting that universal mental health risk screening via 
student self-report is potentially useful for identifying aggregated community risk in a given school 
that may warrant differential deployment of mental health prevention and intervention strategies. 
BESS results reliably identified individual mental health risk associated with special education 
placement, which is documented to lead to poor school outcomes such as school dropout and lack 
of enrollment in post-secondary education. [West J Emerg Med. 2013;14(4):] 
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INTRODUCTION 

Child and adolescent mental health disorders are known 
to be associated with, or increase the risk for, numerous poor 
school and life outcomes for children and adolescents, including 
suicidal ideation and attempts, academic underachievement 
and school dropout, substance use and disorders, and physical 
fighting or victimization by a weapon.' * A preventive approach 
to mitigating associated impairment, morbidity, and poor 
outcomes in school settings has been advised for at least 4 
decades.' The widespread adoption of preventive models, 
methods, and procedures for achieving this goal, however, has 
remained nascent in U.S. schools.* 

Schools have long been identified as the community 
context of choice for delivering preventive mental health 
services. As major societal institutions, schools provide an 
organizational structure that reaches more children with 
more continuity than primary care, or any other child and 
family service setting.' Schools, however, are for the most 
part unprepared to provide preventive mental health services 
due to limited staff training, a competing focus on and time 
commitment to educational service delivery, and a lack of 
assessment methods for delivering services such as universal 
screening, which is a necessary component of any preventive 
mental health service delivery program.**- 

Universal screening is the first step in any preventive, 
secondary prevention, or early intervention program for 
mental health problems.' A National Academies of Sciences 
report identified 4 levels of prevention including, 1) universal 
prevention where community risk factors, such as school safety, 
are of interest, 2) selective prevention where high risk groups, 
such as children exposed to maternal depression, are identified 
for services, 3) indicated prevention where screening for 
behavioral and sub-syndromal symptoms is used to identify 
children for early intervention services (defined as behavioral 
or emotional risk (BER), for the purposes of this study), and 
4) assessment for detection, diagnosis, and treatment of a 
mental health disorders.^ In this report, the goals of universal 
screening were defined as: 

The goals and design of these initiatives should be 
targeted to relatively narrow and specific purposes, for 
example, (1) improving school success for struggling 
students, (2) preventing bullying and student harassment, 
(3) improving teacher and peer relationships, (4) 
increasing school safety and security, or (5) learning to 
regulate and control behavior, (p. 230) 

Although the results of the study could inform all of these 
goals, the school administrators in the present study requested 
surveillance screening for risk factors associated with goals 
3 and 4, due to their concerns about an overall level of 
misbehavior in some schools that was adversely affecting 
teacher and student morale and culminating in incidents 
of violence in the schools. In one of the high schools in 
particular, several incidents of student violence on campus 
caused alarm and growing concerns about the safety of the 
students and teachers. 



A central impediment to the adoption of universal 
screening measures for school-based screening of large 
groups of children has been the practicality of such 
measures, especially the associated personnel costs and test 
administration time that competes directly with the demand 
for academic instructional time.'" Although newer screening 
measures such as the one used in this study require only a few 
minutes per child, the practicality of screening thousands of 
adolescents in numerous schools is yet to be determined." '^ 
However, emerging evidence suggests that some of the 
barriers to feasibility can be overcome with administrative 
support and planning prior to implementing a universal 
screening program.'^ 

Behavioral problem surveillance measures, such as the 
Youth Risk Behavior Survey (YRBS), have been in use for 
some time."'"' This measure assesses student perceptions of 
the frank expression of behavior problems (e.g. smoking, 
sexual activity, etc.), rather than risk per se." ''* Mental 
health risk or BER tests represent a relatively newer class 
of measures that assess a different construct, the predictors 
of mental health disorders, special education placement, or 
behavioral problems such as those assessed on the YRBS or 
its variants, including the California Healthy Kids Survey 
(CnKS).''-'" A recent study by Dowdy and colleagues provides 
empirical evidence of the difference between the YRBS and 
BER constructs. They found that supplementing the CHKS 
with a self-report measure of sub-syndromal BER, the same 
measure used in the current study, significantly increased 
the prediction of cigarette, marijuana, and alcohol use, binge 
drinking, physical fighting, threatening or injury by a weapon, 
skipping school, and serious consideration of suicide as 
measured by the CHKS.'* 

Based on this evidence, we expected a self-reporter 
screener to be a useful tool for assessing school-level BER. 
Thus, the current investigation sought to determine: 

1 . Whether or not a brief adolescent self-report screener 
of BER could be used universally in middle and high 
school with little concern about interference with 
instructional time or other practical concerns. 

2. If the screener would produce score differences between 
schools that were consistent with school administrator 
concerns, which predicted that some schools were 
characterized by more adolescent BER than others. 

3. Whether or not demographic variables such as child 
race/ethnicity, gender, SES, or grade level were strongly 
associated with screener scores. 

4. If individual screener results demonstrated discriminant 
validity by assessing their association with classification 
as eligible for special education programs due to the 
presence of severe behavioral and emotional problems 
or diagnosed mental health disorders. 

METHODS 
Sample 

Data were collected from 3 middle and 4 high schools 
in a mid-sized city in the Southeastern United States. The 
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Table. Descriptive statistics: factor score means and standard deviations for 4 Behavioral and Emotional Screening System factors by school. 



School 






Hyperactivity 


Internalizing 


School problems 






Adjustment factor 


Factor 


Factor 


Factor 


IVliddle school 1 


IVlean 


3.3684 


1.7209 


1.6852 


1.7966 




N 


147 


147 


146 


147 




Standard deviation 


0.50167 


0.51180 


0.49201 


0.55487 


IVIiddle school 2 


IVlean 


3.2468 


1.6737 


1 .7022 


1.6825 




N 


95 


95 


95 


95 




Standard deviation 


0.53925 


0.47835 


0.51825 


0.47582 


IVliddle school 3 


IVlean 


3.1277 


1.8622 


1.8493 


1.9603 




N 


105 


105 


105 


105 




Standard deviation 


0.58581 


0.54774 


0.53887 


0.60221 


High school 1 


Mean 


3.2878 


1.8114 


1.7980 


1.9635 




N 


205 


205 


205 


205 




Standard deviation 


0.53559 


0.52631 


0.54994 


0.57015 


High school 2 


Mean 


3.3764 


1.6854 


1.6355 


1 .9460 




N 


368 


368 


368 


368 




Standard deviation 


0.47221 


0.46557 


0.45389 


0.51496 


High school 3 


Mean 


3.2300 


1.8855 


1.7607 


1.9912 




N 


849 


849 


848 


849 




Standard deviation 


0.54561 


0.55194 


0.49715 


0.55366 


High school 4 


Mean 


3.3151 


1.8172 


1.7385 


1.9836 




N 


453 


453 


453 


453 




Standard deviation 


0.50231 


0.54746 


0.46482 


0.51579 


Total 


Mean 


3.2820 


1.8105 


1.7356 


1.9521 




N 


2222 


2222 


2220 


2222 




Standard deviation 


0.52681 


0.53454 


0.49409 


0.54514 



school district requested universal screening for each of 
the participating schools because of concerns that student 
misbehavior was beyond the typical rate. At the time of 
screening, the school district had been recently cited for 
substandard student academic achievement. In addition, several 
incidents of school violence had occurred on the campus of 
one of the high schools. The sample collected included 2,222 
adolescents in the eighth through 12* grades, ranging from 
198 in grade 12 to 725 in grade 9. Females constituted 53% of 
the sample. The ethnic/racial group sample sizes were 1,701 
African-American, 456 White, 32 Latino, and 26 Other. Sample 
sizes by school ranged from 95 to 849 (median = 205). The 
school district has a high poverty rate, with 76% of students 
in the present study being eligible for a free or reduced- price 
lunch. Approximately 6% of the sample, or 139 adolescents, 
were classified as special education eligible. 



Procedure and Instrument 

The Behavioral and Emotional Screening System (BESS) 
Student Form was administered to all students in groups, 
usually in homerooms, by school district employed school 
psychologists and school psychology doctoral students. 
Administration amounted to approximately 1 5 minutes per 
classroom, including the reading of instructions from a script, 
completion of forms, and collecting forms from all students. 
Total administration was less than 1 hour total per middle 
or high school since all data were collected at the same time 
interval, in most cases during the homeroom period. The 
data collection, entry, cleaning, file preparation, and analyses 
were either conducted or supervised by a post-doctoral fellow 
supported by the Georgia Measurement and Assessment 
Training (GMAT) program (funded by grant number 
R324B080D06 from the Institute of Education Sciences, US 
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Figure 1. Factor score means for 4 Behavior and Emotional Screening System by school. 
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Figure 2. Factor score means and standard deviations for 4 Behavior and Emotional Screening System factors by student grade. 
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Department of Education). This study was approved by both 
the local school district institutional review board (IRB) and 
the IRB of the university of the lead researcher 

The BESS Student is a brief screening measure completed 
by students to identify BER among youth aged 8 through 1 8 
years." It assesses a wide range of behavioral problems and 
strengths, including internalizing problems, externalizing 
problems, school problems, and adaptive skills. The BESS 
requires no explicit training, has 30 items, and can be 
completed in 5 minutes or less per student. Use of theory and 
factor analysis to develop the measure resulted in the inclusion 
of more internalizing items than other screening measures of 
this length, and a 4-factor solution, including an assessment 
of inattention/hyperactivity, internalizing problems, school 
problems, and personal adjustment."' " 

Students are given 4 rating options — never, sometimes, 
often, or almost always — for each item and the sum of the 
items generates a total J-score with higher scores reflecting 
a higher level of BER.'* The scoring rubric or risk level for 
BER is as follows: (a) a T-score of 20-60 suggests a "normal" 
level of risk; (b) 61-70 suggests an "elevated" level of risk; 
and (c) 71 or higher suggests an "extremely elevated" level of 
risk. The risk level classification cut-scores were developed to 
maximize sensitivity and specificity, and results suggest that 
sensitivity, specificity, positive predictive value, and negative 
predictive value were generally high '"^ 

Student responses on the BESS may be entered by hand 
or via scanner with computer software, as was done for 
this investigation. The software report includes raw scores, 
J-scores, and percentiles based on a normative sample that 
closely matched U.S. Census population characteristics. The 
existence of validity scales has been cited as a particular 
advantage of the BESS by independent reviewers.' 

BESS split-half reliability estimates range from 0.90 to 
0.97. Test-retest reliability estimates are high, ranging from 
0.80 to .91. Inter-rater reliability estimates range from 0.71 to 
0.83. The concurrent validity of the BESS was examined by 
administering the items with other social-emotional measures: 
Achenbach System of Empirically Based Assessment 
(ASEBA, 0.71-0.77), Conners' Rating Scales (CRS, 0.51- 
0.78), and Children's Depression Inventory (CDI, 0.51). 

RESULTS 

As stated in the procedure, the BESS self-report screener 
of BER was administered universally in 3 middle and 4 
high schools with little interference with instructional time. 
Therefore, there was some evidence to support our first 
hypothesis that screening could be carried out practically and 
efficiently in schools. 

Descriptive statistics for the sample by school are 
shown in the Table. In order to test whether the screener 
would produce score differences between schools that were 
consistent with school administrator concerns, an Analysis 
of Variance (ANOVA) comparing schools was conducted. 
This ANOVA revealed statistically significant differences 
in the 4 factor scores among the 7 schools with F values 
ranging from 4.73 for the Internalizing factor to 8.20 for the 



Hyperactivity factor, and all tests being significant beyond 
p < 0.001. These results supported the a priori predictions 
of the school administrators in that the high school that had 
experienced increases in school violence (High School #3) 
and its feeder middle school (Middle School #3) had the 
highest means on the deviant factors and the lowest means for 
the Adjustment factor. However, as demonstrated in Figure 
1, these statistically significant differences across school by 
factor were small. 

The third question of interest was whether or not 
demographic variables, such as child race/ethnicity, gender, 
SES, or grade level, were strongly associated with screener 
scores The ANOVA by child gender was statistically 
significant for only the Adjustment {F = 6.79, p < 0.009) 
and Internalizing factors {F= 2535, p < 0.000), where girls 
obtained higher scores on each. BESS scores also differed 
significantly by grade level for all four factors; Adjustment 
iF=3.S7,p< 0.004), Inattention/Hyperactivity (F= 6.23, 
p < 0.001), Internalizing {F=2.S2,p< 0.24), and School 
Problems {F = 6.99, p < 0.000). Figure 2 plots means for 
the 4 factors by grade level. These data reveal a trend for 
self-reported problems to increase in high school over levels 
reported by eighth graders. 

Socioeconomic status produced the most non-significant 
findings in that free or reduced lunch eligibility status, 
unlike the other demographic variables, did not produce 
any statistically significant differences between the BESS 
factors. BESS results for student ethnicity, by contrast, did 
differ for the African American and White groups. All four 
BESS factors differed for these 2 groups (p <0.000) with 
White students reporting more deviance and fewer positive 
adjustment skills. The results for the other 2 groups were too 
small to interpret with confidence. 

In relationship to our fourth research question, special 
education status was linked statistically to only 2 of the 
BESS factors; Adjustment (F= 60.10,;? < 0.001) and 
Internalizing (F = 47.30, /» < 0.001). Special education status 
was not a significant predictor of Inattention/Hyperactivity 
or School Problems in the present study. 

All of these analyses were repeated using BESS 
classification scheme as the outcome variable. The BESS 
BER classification system as noted earlier has 3 levels: 
normal, elevated, and extremely elevated risk. These 
results were similar to those found using factor scores. For 
example, no differences were found between BESS factors 
by SES, more White adolescents were classified as elevated 
and extremely elevated in risk than African- American 
adolescents, and girls (13.5%) acknowledged a higher 
prevalence rate of risk than boys (1 1.5%). In comparisons 
where ANOVA results were significant, all Chi Square 
tests of differences in proportions were also statistically 
significant. 

DISCUSSION 

This study suggests that student self-report screening 
results may provide schools and community stakeholders 
with systematic data about mental health risk that may be 
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used to both address and monitor the mental health needs 
of adolescents in school. These data may also be used, as is 
the case with the BESS analytic software, to disaggregate 
BER status for different levels of analysis, and prevention 
and intervention planning, including the classroom, 
school, sub-district, and school district levels, and even 
region and state levels, if desired. Comparisons can then 
be made across levels of analysis and service delivery to 
determine where significant BER exists, whether there are 
Inattention/Hyperactivity, Internalizing, or School Problems, 
or an absence of strengths as assessed by the Adjustment 
factor. This "targeting" of systems in need of change or 
improvement based on data holds the potential to, for the 
first time, focus prevention and intervention efforts based on 
evidence. 

While a full cost and practicality analysis of this 
measure and methodology was not undertaken for the 
purposes of this study, the ability to gather these individual 
student data from an entire school taking less than 1 hour 
of instructional time per academic year using a relatively 
low-cost screening instrument portends greater practicality 
than has been the case in the past. Brown and Grumet,' for 
example, used clinical interviews to survey suicide risk in 
13 middle and high schools in the Washington, D.C. area. 
They found that 45% of adolescents screened positive 
for "previous suicide attempt or ideation, symptoms of 
depression or anxiety, and/or other emotional problems." 
Their screening program, however, was cumbersome due to 
the use of clinical interviews, and it produced cost estimates 
for the program at about $242 per child, an untenable 
figure for large and small school districts alike. In contrast, 
universal screening using a quick and affordable instrument 
such as the BESS does appear to provide a practical solution 
for gathering self-report BER data universally.'-- 

Furthermore, information gathered by the universal 
screening program implemented in the present study was 
able to provide score differences between schools that 
were consistent with school administrator concerns, which 
predicted that some schools were characterized by more 
adolescent BER than others. Using data to guide their 
decisions, school administrators and stakeholders can use 
the data gathered from a universal screening program to 
drive school and community interventions based on the 
areas of need. Although statistically significant, the practical 
significance of these small differences should be assessed. 
However, as all of the 7 schools in the present study were 
selected due to anticipated elevations in BER, generally 
similar levels of BER across schools was not entirely 
surprising. The small nuances across schools could help 
administrators prioritize the types of interventions necessary, 
and triage schools and individuals in order to best focus the 
limited resources that are available for such intervention. 

Differences in BER were found across demographic 
variables of interest, including gender, grade, and race/ 
ethnicity. Overall, females reported greater internalizing 
problems and lower levels of adjustment. High school 



students, particularly those in the ninth grade, reported lower 
levels of adjustment and higher levels of BER as compared 
to eighth grade students. Therefore, transitional difficulties 
from middle school to high school might help to explain this 
increase in BER. White students reported more BER than 
their African American peers. Future studies should continue 
to examine this finding; White students were the minority 
group in the schools of interest in the present study, so it 
could be that out-group status is more predictive of BER 
than membership in any one particular racial/ethnic group. 
Finally, those in special education reporter higher levels of 
internalizing problems and lower levels of adjustment than 
their peers who were not in special education. 

Good surveillance data are prerequisite for initiating 
public health models of service delivery for children with 
BER inside or outside of school." The BESS and other 
newer instruments may be more suitable for providing 
data due to a variety of factors, including their ability to 
gather BER data for a wide age range (preschool through 
high school) and the availability of student, teacher, and 
parent forms. By contrast, the Youth Behavior Risk Survey 
and other methodologies fail to include a comprehensive 
assessment of sub-syndromal psychiatric symptoms or risk 
factors. The National Survey on Drug Use and Health and 
National Health and Nutrition Examination Survey are both 
collected at age 12 and above, and both include only a small 
sampling of psychiatric questions. 

LIMITATIONS 

The sample sizes for this study among particular 
demographic subgroups were sometimes minimal. Although 
screening was conducted universally, the sampling did not 
include all children in a school due to student absences and the 
exclusion of some classrooms due to disability classifications 
that prevented participation. Sample sizes were not adequate 
to conduct some important analyses, such as including a 
Latino sample in the ethnic group comparisons. 

Due to time limitations only one screener was used. 
Although the BESS has supportive reliability and validity 
evidence, it is relatively new. A clear "gold standard" among 
such school-based screening measures has yet to emerge 
based on consensus use; therefore, more research is needed 
regarding the choice of screening assessment. It is too early 
for a test or few tests to become the standards of practice 
since the use of BER screeners, particularly student self- 
report screeners, is still uncommon in U.S. schools. 

CONCLUSION 

Use of a student self-report screener to identify mental 
health or behavioral and emotional risk among adolescents in 
schools produced results confirming the suspicions of school 
district administrators; one middle and one high school 
displayed more risk than others in the district. Although 
this is not a typical study of discriminant validity for a new 
measure it is a proxy that served the needs of school district 
administrators. This study offers some evidence that newer 
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screeners may allow for greater implementation of mental 
health risk surveillance consistent with well-established 
public health practices and needs. 
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